THE CENTER ON 


STANDARDS & 
ASSESSMENT CSAI Update 
IMPLEMENTATION p 


WestEd@®. cRESST 


New Mexico Kindergarten 


Teachers’ Use of Kindergarten 
Entry Assessment Data 


The Kindergarten Observation Tool (KOT), an observation-based assessment tool designed for use during 
the first 30 instructional days of kindergarten, was implemented across New Mexico in fall of 2016 as the 
states’ kindergarten entry assessment (KEA).' After KOT administration ended, the Center on Standards and 
Assessment Implementation (CSAI) helped to develop and administer a survey aimed to understanding 
how New Mexico kindergarten teachers are using data collected through the KOT. This update provides 
information about the survey that was administered, along with a summary of its key findings. 


New Mexico KOT Data Use Survey 

The survey was provided to 1,447 teachers across the state in December 2016. The survey aimed to gather information 
from New Mexico kindergarten teachers who administered the KOT in fall of 2016. The questions focused around 
three main topics: 


1. How kindergarten teachers are using KOT data; 
2. How kindergarten teachers perceive the impact of concurrent assessment initiatives on KOT administration; and 
3. How kindergarten teachers are communicating about KOT results with families. 


During the three weeks in which the survey was open, 371 teachers responded (all responses were recorded 
anonymously), although teachers did not respond to every question on the survey. Approximately 20 percent of these 
teachers had participated in early stages of KOT development, including a 2014 pilot and a 2015 field test. Of the 
respondents, approximately 51 percent had been teaching kindergarten for more than six years. 


1 For more information about KOT development, visit the summary page on New Mexico within the Pre-K/K assessment tab of the State of the States site feature on 
www.csai-online.org. Also see PED’s KOT website. 
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Opinions About Data Use 

The majority of responding teachers agreed or strongly agreed that it is important to use data to inform instructional 
practice (95%); that the KOT provided a helpful snapshot of students’ skills, knowledge, and behavior at the beginning 
of school (73%); and that the KOT data/report(s) helped them to identify students’ strengths and challenges (69%). See 
Figure 1 for details. 


Figure 1. Teachers’ Opinions Regarding Data Use and the Utility of KOT Data/Report(s) 


| think it is important to use data to inform educational practice. 


2% 51% 44% 


The KOT provided a helpful snapshot of my students’ skills, knowledge, and behavior at the beginning of school. 


3% 


The KOT data/report(s) helped me identify my students’ strengths and challenges. 


55% 14% 


Hi Strongly Disagree Hl Disagree Hl Agree Hi Strongly Agree 


Total responding teachers: 347 


Use of KOT Data? 


When asked about their use of KEA data in instructional practices, almost half of teachers indicated a moderate or 
extensive use of KOT data in planning for small groups (50%), planning for one-on-one instruction (52%), planning 
with a focus on Literacy (47%), and planning with a focus on Numeracy? (49%). 


Figure 2. Teachers’ Use of KOT Data in Specific Contexts 
Planning for small groups 


38% 12% 


Planning for one-on-one instruction 


37% 15% 


Planning with a focus on Literacy 


37% 10% 


Planning with a focus on Numeracy 


38% 11% 


HH Did not use HH Minimal use Hl Moderate use Hl Extensive use 
Total responding teachers: 337 
2 The survey included a list of 17 different contexts; the four contexts highlighted here represent the contexts in which teachers used KOT data to a relatively greater extent. 


3 Literacy and Numeracy are two of the six learning domains represented on the KOT; in general, teachers did not report using KOT data to the same extent for 
planning with a focus on the other four domains. 
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Kindergarten Observation Tool Application (KOTA) 

The KOTA is an interactive, online application that has been developed to allow teachers to record and submit KOT 
observations and ratings. The KOTA can be accessed via any web-enabled device (e.g., tablets, laptops, cell phones) 

using a secure login. All data submitted via the KOTA is synced automatically to the database and data is attached to 
each child's unique ID. Teachers, school staff, and district administrators can also access various KOT reports via the KOTA 
system, including reports for families. The majority of responding teachers agreed or strongly agreed that the KOTA was 
easy to access (94%), was easy to use (80%), allowed them to examine various types of data at once (84%), and generated 
useful displays (e.g., reports, tables) (72%). 


Figure 3. Teachers’ Impressions Regarding the KOTA 
I can access the KOTA. 


>1% 73% 21% 


The KOTA is easy to use. 


The KOTA allows me to examine various types of data at once. 


2% 74% 10% 


The KOTA generates displays (e.g. reports, tables) that are useful to me. 


5% 60% 12% 


HH Strongly Disagree HM Disagree Hl Agree Hi Strongly Agree 


Total responding teachers: 332 
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Effect of Concurrent Curriculum and Assessment Initiatives 

New Mexico implemented a new kindergarten English language arts (ELA) curriculum and a new formative assessment 
initiative the same year it implemented the KOT statewide. The following questions asked teachers about the impact of 
this dual implementation. The majority of responding teachers indicated that two concurrent curriculum and assessment 
initiatives—Istation°? and a new English language arts (ELA) curriculum®—had little to no impact on their administration 
of the KOT in specific contexts. 


Figure 4. Impact of Implementation of Istation on KOT Administration 


Preparing to administer the KOT 


41% 31% 8% 


Administration of the KOT 


41% 29% 10% 


Fidelity in administration of the KOT (i.e., adherence to the guidance provided at KOT Training) 


40% 29% 7% 


Data entry for the KOT 


39% 28% 10% 


Reviewing data from the KOT 


41% 25% 9% 
Using data from the KOT to inform curriculum and instruction 

39% 27% 8% 
Sharing results from the KOT with parents and families 


44% 22% 9% 


Hl No impact HM Little to no impact Hi Moderate impact MM Extensive impact 


Total responding teachers: 326 


5 Istation provides New Mexico district and charter schools with literacy-focused formative assessments for use with students in kindergarten through third grade. 
Like the KOT, it was implemented statewide in Fall 2016. For more information, visit http://www.istation.com/newmexico . 


6 Many districts across New Mexico instituted a new ELA curriculum in Fall 2016. 
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Figure 5 Impact of Implementation of New ELA Curriculum on KOT Administration 


Preparing to administer the KOT 


24% 10% 


Administration of the KOT 


25% 10% 


Data entry for the KOT 


25% 9% 


Reviewing data from the KOT 


25% 9% 


Using data from the KOT to inform curriculum and instruction 


25% 9% 


Sharing results from the KOT with parents and families 


19% 7% 


HE Noimpact M8 Minimalimpact WiModerateimpact Ml Extensive impact 


Total responding teachers: 326 


Teachers’ Approaches to Communicating with Families about KOT Results 
One survey question focused on teachers’ communication with families about 


KOT results. Approximately half of teachers who responded to this survey Did you communicate with 
‘ ee . . a families about KOT results? 

question (53%) indicated that they communicated with families about 

KOT results. 


Of the teachers who communicated with families, almost three-quarters (71%) 
agreed or strongly agreed with the statement that the KOT data/report(s) 
enhanced their communication with families. The majority of responding 
teachers (61%) indicated that they communicated with families about KOT 
results during in-person meetings/conferences, using the Family-Teacher 
Summary Report.’ 


Total responding teachers: 331 


7 The Family-Teacher Summary Report is a version of the Individual Student Report. It is specifically designed for teachers to use in communicating with families about 
KOT results, and it provides space for teachers to add notes about their observations of students, as well as space to add family input. 
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Figure 6 Impact of Implementation of New ELA Curriculum on KOT Administration 


The KOT data/report(s) enhanced my communication with families. 


4% 64% 7% 


Hl Strongly Disagree Hl Disagree Hl Agree Hl Strongly Agree 


In what way(s) did you communicate with families? 


38% 27% 
3% 
Hi Sent home a note about the KOT results that Hl Using the Family-Teacher Summary Report, talked 
| wrote myself about individual students’ KOT results 
I Other with families during in-person meetings 
EH Shared Family Brochure with my families Hl Without using the Family-Teacher Summary Report, 


talked about individual students’ KOT results with 


HH Sent individualized Family-Teacher Summary families during in-person meetings 


Reports to each family’s home 


Total responding teachers: 173 of the 174 that communicated with families about the KOT results 


Open-Ended Response Analysis 
Teachers had the option of responding to an open-ended question to outline their use of KOT data. Common themes 
that emerged from the set of open-ended responses include: 


¢ Teachers believe that the KOT helps them better understand what their students know and can do. 

¢ Many teachers expressed concerns about the time that completing KOT data collection takes. However, teachers 
recognize that their concerns about time can be attributed to the fact that the KOT is a new assessment and is 
unfamiliar to them. 

¢ Teachers feel like they did not use KOT data as strategically as they could have and see room to grow in 
subsequent years. 


¢ Teachers see some redundancy between the KOT and other district-required assessments. 
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In addition to these themes, many teachers saw the KOT as an assessment that showed student growth throughout 
the kindergarten year. This is notable because the KOT was designed as a point-in-time assessment (at kindergarten 
entry), and was not designed to be a metric of growth over time. The New Mexico Public Education Department's KOT 
website clearly states the purpose of the assessment: 


ose of the KOT is to gather information about what 
i know and can do at the time of kindergarten entry. 
The KOT will serve as a ‘bridge’ between preschool and grade 1.” 


Considerations for States with KEAs 
The results of this survey can be used to inform states’ decisions about how best to provide support and professional 
development opportunities to kindergarten teachers. 


Analysis of the open-ended responses provides strong evidence that, in general, teachers would benefit from 
additional opportunities to: 


¢ Build their assessment literacy and understanding of the purpose of KEAs; 

¢ Learn strategies for using KEA data to improve instruction and curriculum; and 

¢ Participate in collaborative experiences, such as structured meetings or focus groups, where they could discuss 
and reconcile concerns about assessment time and the challenges of balancing the demands of multiple 
assessment requirements at the state and local levels. 


Additional ways that schools, districts, and the state can support teachers in using KEA data more effectively include: 


¢ Establishing a culture of data use to ensure that data-based decisions are made frequently, consistently, and 
appropriately; 

¢ Creating an overall plan for KEA data that includes a common vision and shared goals; 

¢ Providing ongoing data leadership at the school and district levels that serves as a resource for educators; and 

¢ Encouraging data use by investing in regular professional development and structured time for collaboration, 
reviewing data, and discussing data.; 


KEAs can bring together a combination of data and anecdotal information about students, how they learn, and their 
educational environments. Through continued work with educators, states can help reduce barriers to data use and 
ensure that educators are using and communicating valuable information about their students’ learning. 
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